An Empathic-tutoring System Using Spoken Language

This paper represents an approach to simulate the mental activities of children during their interaction with computers through their spoken language. The mental activities are categorized into three states: confidence, confusion and frustration. Four knowledge sources are used in the detection. One is prosody, which is indicative of utterance type and user’s attitude. The second is embedded key words/phrases which help interpret the utterances. Moreover, it is found that children’s speech exhibits very different acoustic characteristics from adults. Given the uniqueness of children’s speech, this paper applies a vocal-tract-normalization (VTLN)-based technique to compensate for both inter-speaker variability and intra-speaker variability in children’s speech. The detected key words/phrases are then integrated with prosodic information as the cues for the MAP decision of mental states. Tests on a set of 50 utterances collected from the project preliminary experiments showed the classification accuracy was %.